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Abstract 

OpenGM is a C-l — I- template library for defining discrete graphical models and performing 
inference on these models, using a wide range of state-of-the-art algorithms. No restrictions 
are imposed on the factor graph to allow for higher-order factors and arbitrary neighbor- 
hood structures. Large models with repetitive structure are handled efficiently because 
(i) functions that occur repeatedly need to be stored only once, and (ii) distinct functions 
can be implemented differently, using different encodings alongside each other in the same 
model. Several parametric functions (e.g. metrics), sparse and dense value tables are pro- 
vided and so is an interface for custom C++ code. Algorithms are separated by design 
from the representation of graphical models and are easily exchangeable. OpenGM, its 
algorithms, HDF5 file format and command line tools are modular and extendible. 
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1. Introduction and Related Work 

Graphical models have become a standard tool in machine learning, and inference (marginal 
and MAP estimation) is the central problem, cf. Nowozin and Lampert (2011). 

These models can be defined rigorously as models of functions that factorize w.r.t. an 
associative and commutative operation, cf. Werner (2008). The C++ library OpenGM is 
based on this general definition that allows for a unified treatment of accumulative opera- 
tions on such functions, including optimization, summation (marginalization) , conjunction 
and disjunction. It provides a variety of inference algorithms 1 beyond message passing 
(Fig. 1). It can deal efficiently with large scale problems, since (i) functions that occur 
repeatedly need to be stored only once and (ii) when functions require different parametric 
or non-parametric encodings, multiple encodings can be used alongside each other, in the 
same model. No restrictions are imposed on the factor graph and the operations of the 
model, and the file format handles user extensions automatically. Furthermore, OpenGM 
is a template library in which elementary data types can be chosen to maximize efficiency. 

Existing libraries do not have all of these properties. MRF-lib (Szeliski et al., 2008) 
is restricted to the min-sum semi-ring and second-order grid graphs. While it is highly 
efficient on these models, it is also specialized to these and not easily extendible. In contrast, 
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Figure 1: Algorithms provided by OpenGM: Loopy BP (Pearl, 1988; Kschischang et al., 2001), 
TRBP (Wainwright and Jordan, 2008), TRW-S (Kolmogorov, 2006), a-expansion, a/3-swap (Boykov 
et al., 2001), QPBO (Rother et al., 2007), ICM (Besag, 1986), Lazy Flipper (Andres et al., 2010), 
LOC (Jung et al., 2009), Swendsen-Wang sampling (Barbu and Zhu, 2005), Dualdecomposition 
(subgradient and bundle-methods) (Kappes et al., 2012; Komodakis et al., 2011), native LP and 
Branch & cut using IBM ILOG Cplcx, A* (Bergtholdt et al., 2010). 



libDAI (Mooij, 2010) supports max-product and sum- product semi-rings which are hard- 
coded. The main drawback of libDAI is that it supports only dense value tables to encode 
functions which becomes prohibitive for models with many labels and higher-order factors. 
Similar to libDAI, Fastlnf (Jaimovich et al., 2010) focuses on message passing and does 
not impose any restrictions on the factor graph. In contrast to libDAI, it supports shared 
functions and different function types in a so-called relational model that is similar in 
spirit to the design of OpenGM. However, Fastlnf supports only sum-product semi-rings 
and, unlike OpenGM, has no generic template abstraction of semi-rings. The recently 
published library grante (Nowozin, 2012) provides shared functions and different function 
types. Furthermore, it comes with a set of learning methods. Unlike OpenGM, it is not 
template based, limited in its inference methods and published under a proprietary license. 

The generality of OpenGM comes at the cost of performance. And yet, OpenGM is only 
slightly slower than libDAI when running loopy belief propagation on a grid graph. The 
highly optimized code of MRF-LIB is twice as fast for general second-order factors and 20 
times as fast for standard metrics. 

OpenGM is modular and extendible. The graphical model data structure, inference 
algorithms and different encodings of functions interoperate through well-defined interfaces. 



2. Mathematical Foundation 

OpenGM is built on a rigorous definition of the syntax and semantics of a graphical model. 
The syntax determines a class of functions that factorize w.r.t. an associative and com- 
mutative operation. In a probabilistic model, it determines the conditional independence 
assumptions. The semantics specify the operation and one function out of the class of all 
function that are consistent with the syntax. 

The syntax (Fig. 2a) consists of a factor graph, i.e. a bipartite graph (V, F, E), a linear 
order < in V , a set I whose elements are called function identifiers, and a mapping 7 : F — > I 
that assigns one function identifier to each factor such that only factors that are connected 
to the same number of variables can be mapped to the same function identifier. 

For any v G V and / 6 F, the factor f is said to depend on the variable v iff (v, /) € E. 
Af(f) denotes the set of all variables on which / depends and (v- )je{i,...,\j\f(f)\} the sequence 
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a. Syntax 



X V1 = X V2 = X V3 = {0, . . . , n} =: X v 

<Ph > <Ph > : A"„ — > f2, 99j 4 : A 2 — >■ $7 

<p : — > Q, such that V(x Vl ,x V2 ,x V3 ) £ X^: 

( f{x Vl , X V2 , X.y 3 ) = (Xjjj )(^9j 2 (^^2 )c/3j 3 (X{, 3 ) 

^14 (X-Ul ) 2+2 ) V«4 (^+2 1 -E'U 3 ) 

b. Semantics 



Figure 2: A factor graph (V,F,E) describes how a function ip decomposes into a product 
of functions. In OpenGM, we extend this syntax by a set I of function identifiers and 
a mapping 7:^—7-/ that assigns one function identifier to each factor. In the above 
example, the factors and /s are mapped to the same function identifier 14, indicating 
that the corresponding functions (fi 4 (x Vl , x V2 ) and <Pi 4 (x V2 ,x V3 ) are identical. 



of these variables in ascending order. Similarly, {vj)je{i,...,\v\} denotes the sequence of all 
variables in ascending order. 

Semantics (Fig. 2b) w.r.t. a given syntax consist of one finite set X v 7^ for each v £ V, 
a commutative monoid (0,0, 1) and for any i £ I for which there exists an / £ F with 
j(f) = i, one function 2 ipi : Xjj) x • • • x Xjj) — > VL. 

The function from X := X v 
function ip : X — > Q such that \/(x 



■ x X v . to Q induced by syntax and semantics is the 
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W.l.o.g., OpenGM simplifies the syntax and semantics by substituting V = {0, . . . , |V| — 
1}, equipped with the natural order, F = {0, . . . , |F| — 1}, / = {0, . . . , |/| — 1} and for each 
v £ V, X v = {0, . . . , \X V \ — 1}. A graphical model is thus completely defined by the number 
of variables |V|, the number of labels \X V \ of each variable v £ V, the edges E of the 
factor graph, the number of functions \I\, the assignment of functions to factors 7, the 
commutative monoid (0,0, 1) and one function ipt for each function identifier i £ I. 

Given a graphical model and, instead of just the commutative monoid (f2, 0, 1), a com- 
mutative semi-ring (O, 0, 1, ©, 0), the problem of computing 

0¥>(s) i-e- ©G> 7 (/) -> x v$ lf J ^ 

xeX x&X fdF v 

is a central problem in machine learning with instances in optimization (M, +, 0, min, 00), 
marginalization (M + , •, 1, +, 0) and constrained satisfaction ({0, 1}, A, 1, V, 0). 



3. Using and Extending OpenGM 

The first step when using OpenGM is to construct a label space that determines the number 
of variables and the number of labels of each variable. The next step is to fix the data type 



2. The existence of V implies V/, f € F : 7 (/) = 7 (/') => Vj G {1, . . . , deg(/)} : X Jf) = X 
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of the domain f2, the operation and the way functions are encoded, by choosing the 
parameters of the graphical model class template as in the example below. 

To define a function such as <pi 4 (yi, 2/2), one needs to indicate how many labels y\ and 2/2 
have and set the parameters of the function or fill its value table. Once a function has been 
added to the model, it can be connected to several factors and thus assigned to different 
sets of variables. This procedure is always the same, regardless of the number and type of 
classes used to encode functions. Details are described in the users' section of the manual. 

Algorithms for optimization and inference are classes in OpenGM. To run an algorithm, 
one instantiates an object of the class, providing a model and optional control parameters, 
and calls the member function infer, either without parameters or with one parameter 
indicating a visitor (see example) . Visitors are a powerful tool for monitoring and controlling 
algorithms by code injection. Once an algorithm has terminated, results such as optima 
and bounds can be obtained via member functions. Detailed instructions can be found in 
the users' section of the manual. 

OpenGM provides interfaces for custom algorithms, custom parametric functions, cus- 
tom discrete spaces and custom semi-rings. These interfaces are described in the developers' 
section of the manual. 

1 typedef SimpleDiscreteSpace<size_t, size_t> Space; 

2 Space space(numberOfVariables, numberOfLabels); 

3 typedef OPENGM_TYPELIST_2(ExplicitFunction<float>, PottsFunction<float>) Functions; 

4 typedef GraphicalModel<float, Adder, Functions, Space> Model; 

5 Model gm(space); 

6 ExplicitFunction<float> fl(&numberOfl_abels, &numberOfl_abels + 1); 

7 fl(0) = ...; fl(l) = ...; ... 

8 Model::Functionldentifier fidl = gm.addFunction(fl); 

9 const size_t variablelndex = 0; 

10 gm.addFactor(fidl, ^variablelndex, &variablelndex + 1); 

11 PottsFunction<float> f2(numberOfl_abels, numberOfLabels, O.Of, 0.3f); 

12 Model::Functionldentifier fid2 = gm.addFunction(f2); 

13 size_t variablelndices[] = {variablelndex, variablelndex + 1}; 

14 gm.addFactor(fid2, variablelndices, variablelndices + 2); 

15 typedef BpUpdateRules<Model, Minimizer> UpdateRules; 

16 typedef MessagePassing< Model, Minimizer, UpdateRules, MaxDistance> BeliefPropagation; 

17 BeliefPropagation::Parameter parameter(maxNumberOflterations, convergenceBound, damping); 

18 BeliefPropagation bp(gm, parameter); 

19 MessagePassingVerboseVisitor<BeliefPropagation> visitor; 

20 bp.infer(visitor); 

21 vector<size_t> labeling(numberOfVariables); 

22 bp.ar g(l abe,ing); 

4. Conclusion 

OpenGM is a C++ library for finite graphical models that provides state-of-the-art inference 
algorithms. It widens the range of models representable in software by allowing for arbitrary 
factor graphs and semi-rings and by handling models with repetitive structure efficiently. It 
is fast enough for prototype development even in settings where performance is paramount. 
The modularity and extendibility of OpenGM, its command line tools and file format have 
the potential to stimulate an exchange of models and algorithms. 
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